In recent years, large language models have demonstrated remarkable performance across various natural language processing (NLP) tasks. However, deploying these models for real-world applications often requires efficient inference solutions to handle the computational demands. In this paper, we explore the utilization of CPUs for accelerating the inference of large language models. Specifically, we introduce a parallelized approach to enhance throughput by 1) Exploiting the parallel processing capabilities of modern CPU architectures, 2) Batching the inference request. Our evaluation shows the accelerated inference engine gives an 18-22x improvement in the generated token per sec. The improvement is more with longer sequence and larger models. In addition to this, we can also run multiple workers in the same machine with NUMA node isolation to further improvement in tokens/s. Table 2, we have received 4x additional improvement with 4 workers. This would also make Gen-AI based products and companies environment friendly, our estimates shows that CPU usage for Inference could reduce the power consumption of LLMs by 48.9% while providing production ready throughput and latency.

The rapid advances in artificial intelligence (AI) as demonstrated by the recent launch of GPT-4 and previously by ChatGPT are generating a great deal of excitement. Artificial intelligence continues to evolve by offering new possibilities in various industries and aspects of human existence, creating numerous debates about its potential impact on our everyday lives and across the global economy. The C-Suite of large organisations in different sectors are actively discussing whether and how such models may be deployed within their organisations, whilst at the same time there has been a rapid adoption of the models by end users. However, Large Language Models (LLMs) using Transformers with the self-attention mechanism is not the only area of AI that is advancing rapidly. Alongside the vast potential of LLMs and the Transformer based approach that underlies it, is also the rise of the AI on the Edge (of the network), across the devices that we interact with in our daily lives.

In a country such as India that has a low doctor-patient ratio, Artificial Intelligence (AI) can enable greater access to expert care from anywhere, with telehealth and robotics applied across inpatient and outpatient environments. Experts says AI will bolster the role of healthcare by assisting in screening, diagnosis, and treatment of diseases thereby improving quality of life and reducing the cost burden for patients. "India has a tremendous opportunity to lead human-centric applications and democratise AI for the world backed by high skilled talent, technology, vast data availability, and the potential for population-scale AI adoption," says Vice-president and managing director of Sales, Marketing and Communications Group, Intel India. Intel has been focusing its efforts towards accelerating AI innovation to deliver transformative healthcare solutions and democratise healthcare access and delivery in India. The company's portfolio of compute, memory, storage, and networking technologies powers some of the most exciting healthcare and life sciences applications.

In a country such as India that has a low doctor-patient ratio, Artificial Intelligence (AI) can enable greater access to expert care from anywhere, with telehealth and robotics applied across inpatient and outpatient environments. Experts says AI will bolster the role of healthcare by assisting in screening, diagnosis, and treatment of diseases thereby improving quality of life and reducing the cost burden for patients. "India has a tremendous opportunity to lead human-centric applications and democratise AI for the world backed by high skilled talent, technology, vast data availability, and the potential for population-scale AI adoption," says Prakash Mallya, vice-president and managing director of Sales, Marketing and Communications Group, Intel India. Intel has been focusing its efforts towards accelerating AI innovation to deliver transformative healthcare solutions and democratise healthcare access and delivery in India. The company's portfolio of compute, memory, storage, and networking technologies powers some of the most exciting healthcare and life sciences applications. The cloud-based AI solution Netra.AI is the latest example of the impact and innovation that can be made possible with Intel technology.

A new deep learning model can be a solution for diabetic retinopathy. Sankara Eye Foundation and Leben Care, a Singapore-based company are collaborating to provide diabetic retinopathy relief in India. The companies are working on an Intel-powered, cloud-based artificial intelligence solution that uses deep learning to detect retinal conditions within a short span. Called Netra.AI, the accuracy of this system matches that of human doctors and can drastically reduce screening burdens. Prakash Mallya, vice president and managing director of sales, marketing, and communications group at Intel India says, "The use of AI to improve disease detection and prevention is a critical step for the healthcare industry and a giant leap for humankind. India has one of the largest diabetic populations in the world and diabetic retinopathy is the major cause of vision loss and blindness in persons of working age. WIth Netra.AI, Sankara Eye Foundation, and Leben Care have leveraged the power of Intel Xeon Scalable processors and built-in Intel Deep Learning (DL) Boost to accurately detect DR and enable timely treatment to effectively combat avoidable vision impairment and blindness in diabetic patients."

Sponsored Artificial intelligence (AI) promises to revolutionize healthcare. The underlying combination of Machine Learning and analytics can process medical data sets so large and medical images so numerous that they are beyond the scale of researchers, physicians and staff. In so doing, this AI duo promises to help identify patients at risk and prevent the onset of diseases and medical conditions. For existing patients, the hope is AI can identify hidden illnesses, pinpoint medical problems and in the development and application of treatments that assist patient recovery. Yet adoption has been held back thanks to the cost and complexity of building and owning the kinds of high-performance systems needed.

If you haven't committed to a comprehensive AI strategy, your competitors might already have an unfair advantage. At the New York Times DealBook conference, Intel emphasized it was urgent that every company put an artificial intelligence (AI) strategy in place. The reason, in a word, is data. The data deluge continues to accelerate, with data points from the Internet of Things (IoT) alone expected to bring another 20 billion new sources of information within the next two years. The amount of data coming into businesses exceeds the capability of humans to process it all.

They also demand increased flexibility with hardware that allows them to program with mainstream languages at a higher abstraction level along with libraries. The data science community is looking for a complete solution stack that abstracts away the hardware specifics, allowing them the ease to crunch parallel workloads more efficiently.

Accelerate your HPC and AI workloads with new products, advanced technologies, and services from HPE. A growing number of commercial businesses are implementing HPC solutions to derive actionable business insights, to run higher performance applications and to gain a competitive advantage. In fact, according to Hyperion Research, the HPC market exceeded expectations with 6.8% growth in 2018 with continued growth expected through 2023.1 Complexities abound as HPC becomes more pervasive across industries and markets, especially as you adopt, scale and optimize HPC and AI workloads. HPE is in lockstep with you along your AI journey. We help you get started with your AI transformation and scale more quickly, saving time and resources.

This feature continues our series of articles that survey the landscape of HPC and AI. This final post explores AI hardware options to support the growing artificial intelligence software ecosystem. Balance ratios are key to understanding the plethora of AI hardware solutions that are being developed or are soon to become available. Future proofing procurements to support run-anywhere solutions--rather than hardware specific solutions--is key! The basic idea behind balance ratios is to keep what works and improve on those hardware characteristics when possible.